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ABSTRACT 


A challenge in digital learning games is assessing students’ 
learning behaviors, which are often intertwined with game 
behaviors. How do we know whether students have learned 
enough or needed more practice at the end of their game 
play? To answer this question, we performed post hoc anal- 
yses on a prior study of the game Decimal Point, which 
teaches decimal numbers and decimal operations to mid- 
dle school students. Using Bayesian Knowledge Tracing, we 
found that students had the most difficulty with mastering 
the number line and sorting skills, but also tended to over- 
practice the skills they had previously mastered. In addition, 
using students’ survey responses and in-game measurements, 
we identified the best feature sets to predict test scores and 
self-reported enjoyment. Analyzing these features and their 
connections with learning outcomes and enjoyment yielded 
useful insights into areas of improvement for the game. We 
conclude by highlighting the need for combining traditional 
test measures with rigorous learning analytics to critically 
evaluate the effectiveness of learning games. 


Keywords 
Decimal, Digital Learning Game, Bayesian Knowledge Trac- 
ing, Over-practice 


1. INTRODUCTION 


Digital learning games are typically regarded as a power- 
ful tool to promote learning by engaging students with a 
novel and interactive game environment. While there have 
been concerns about the lack of empirical results on learn- 
ing games’ effectiveness [21,32], recently we have seen more 
research that addresses this issue by showing students’ learn- 
ing gains from pretest to posttest in rigorous randomized ex- 
periments [9, 41,52]. More generally, a meta-analysis of 69 
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studies by [10] showed that game conditions promoted signif- 
icantly more learning than non-game conditions with equiv- 
alent knowledge content, and that augmented game designs 
with more learning-oriented features were more instruction- 
ally effective than standard designs. 


While this prior research has demonstrated that digital learn- 
ing games can enhance learning, the next step is to exam- 
ine how they do so. In particular, even though the common 
measures of pretest and posttest scores are necessary to eval- 
uate students’ transferable learning, they are inadequate to 
address many questions about how learning takes place dur- 
ing the game. For example, did students get just enough 
practice from the game, or more practice than necessary? 
How does in-game learning correlate with test performance? 
These questions have been explored in great detail in Intel- 
ligent Tutoring Systems (ITS), but not as much in digital 
learning games, primarily because of the differences in design 
approaches between these two platforms. ITS are typically 
very structured environments where students are frequently 
evaluated on their knowledge and, in the mastery learning 
settings [28], move to a new skill as soon as the system de- 
termines they have mastered the current skill. In contrast, 
digital learning games emphasize students’ freedom in shap- 
ing their own learning experience without concern about the 
consequences of failure [15]; as a result, the game’s learning 
objectives are not always obvious to the students [4]. The 
question, then, is how can we combine the traditional pretest 
and posttest measures in learning game studies with learn- 
ing analytics methods from ITS to paint a better picture of 
students’ learning, both inside and outside of the game con- 
text? Furthermore, given the game’s dual goal of promoting 
both learning and enjoyment, do in-game learning metrics 
also relate to students’ enjoyment in any meaningful way? 


Our work explores these questions in the context of Decimal 
Point, a game that teaches decimal numbers and operations 
to middle-school students. Here we present a post hoc analy- 
sis of the data from a prior study [22]. First, we investigated 
how well students mastered the in-game skills, how long it 
took them to master each skill, and whether students con- 
tinued practicing after mastery. Next, we used student data 
from before and during game play to predict their learning 
outcomes and enjoyment after the game. Based on this re- 
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sult, we derived lessons for improving learning support in 
Decimal Point as well as in a more general learning game 
context. 


2. RELATED WORK 


2.1 Learning Analytics in Games 

In-game formative assessment can be a powerful comple- 
mentary tool for capturing students’ learning progress [59]. 
Traditional formative measures typically make use of game- 
based metrics, such as the number of completed levels or 
the highest level beaten [2,11], but these metrics may not 
always align with actual learning. Prior studies on Deci- 
mal Point, for instance, reported that students who played 
more mini-game rounds did not learn more than those who 
played fewer [18,39]. An alternative approach is to employ 
learning analytics methods from ITS studies. For exam- 
ple, learning curve analysis, which visualizes students’ error 
rates over time, has been applied in several learning games 
and yielded valuable insights that range from instructional 
redesign lessons to discovery of unforeseen strategy by stu- 
dents [17, 29, 42]. 


Learning analytics techniques can also connect formative as- 
sessment with external performance. For example, Bayesian 
networks have been applied to predict posttest responses 
from students’ in-game data in several learning games [30, 
48,54]. Similarly, [27] employed feature engineering and gra- 
dient boosted random forest algorithm to identify struggling 
students in real-time in a physics learning game. Recently 
we have also seen more usage of deep learning for this predic- 
tion task [24,51]. In general, research work in this direction 
can illustrate how well students’ learning aligns with the 
game’s learning objectives, while also guiding the develop- 
ment of adaptive support game features. 


2.2 Decimal Point 

Decimal Point is a web-based single-player digital learning 
game that helps middle-school students learn about decimal 
numbers and their operations (e.g., adding and comparing). 
The game features an amusement park metaphor, with a 
map of the park used to guide students (Figure 1). There 
are 8 theme areas with 24 mini-games, connected by a line 
that is designed to interleave skill types and theme areas. 
Each mini-game is aimed at helping students solve one of 
the common decimal misconceptions: Megz (longer decimals 
are larger), Segz (shorter decimals are larger), Pegz (the two 
sides of a decimal number are separate and independent) 
and Negz (decimals smaller than 1 are treated as negative 
numbers) [25]. Also, each mini-game calls for one of the 
following skills: 


1. Addition: add two decimals by entering the carry dig- 
its and the sum. 

2. Bucket: compare given decimals to a threshold num- 
ber and place each decimal in a “less than” or “greater 
than” bucket. 

3. Number Line: locate the position of a decimal number 
on the number line. 

4. Sequence: fill in the next two numbers of a sequence 
of decimal numbers. 

5. Sorting: sort a list of decimal numbers in ascending 
or descending order. 


Ancient 
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rf 
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(a) Goal 


(b) Space Raider 


Figure 1: Screenshots of the main map screen and 
two example mini-games. Goal is a Number Line game 
and Space Raider is a Sorting game. 


In each mini-game, students solve a number of decimal prob- 
lems related to the game’s targeted skill and receive imme- 
diate feedback about the correctness of their answers. Stu- 
dents don’t face penalty on incorrect responses and can re- 
submit answers as many times as needed; however, they are 
not allowed to move forward without solving all the prob- 
lems in the mini-game. More details about the instructional 
content of the mini-game problems can be found in [35]. 


The original study of Decimal Point showed that the game 
led to more learning and enjoyment than a conventional tu- 
tor with the same instructional content [35]. Subsequent 
studies have integrated the element of agency into the game, 
by endorsing students to select their preferred mini-games 
to play and stopping time [18,39]. Based on their find- 
ings, students who were provided agency acquired equiva- 
lent learning gains in less time than those who were not. 
Most recently, a study by [22] compared two versions of the 
game, one that encourages students to play to learn, and one 
that encourages them to play for fun. Their results indicated 
that the learning-oriented group focused on re-practicing the 
same mini-games, while the enjoyment-oriented group did 
more exploration of different mini-games. In general, while 
all of these previous works reported that students learned 
from the game across all study conditions, it is not yet clear 
which game factors contributed to these findings. Further- 
more, no connection between students’ learning and their 
enjoyment has been identified. Our work aims at acquiring 
more insights into these areas. 
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Table 1: Survey items before and after game play. 


Pre-intervention surveys 
Dimension (item count) Example statement Cronbach’s a 
Decimal efficacy (3) [44] I can do an excellent job on decimal number math assignments. 83 
Computer efficacy (3) [31] I know how to find information on a computer. 71 
Identification agency (2) [50] | I work on my classwork because I want to learn new things. .60 
Intrinsic agency (2) [50] I work on my classwork because I enjoy doing it. .86 
External agency (3) [50] I work on my classwork so the teacher won’t be upset with me. 61 
Perseverance (3) [12] Setbacks don’t discourage me. I don’t give up easily. 79 
Math utility (3) [13] Math is useful in everyday life. 63 
Math interest (2) [14] I find working on math to be very interesting. 75 
Expectancy (1) [23] I plan to take the highest level of math available in high school. - 
Post-intervention surveys 
Dimension (item count) Example statement 
Affective engagement (3) [5] | I felt frustrated or annoyed. 78 
Cognitive engagement (3) [5] | I tried out my ideas to see what would happen. 54 
Game engagement (5) [7] I lost track of time. TA 
Achievement emotion (6) [43] | Reflecting on my progress in the game made me happy. .89 


3. DATASET 


Our work uses data from 159 fifth and sixth grade students 
in our prior study [22], where students could select and play 
the mini-games from the map in Figure 1 in any order, and 
were allowed to stop playing at any time after finishing 24 
mini-game rounds. They could also play more rounds of 
the completed mini-games, with the same game mechanics 
but different question content. For example, the first round 
of the mini-game Goal asks students to locate 0.76 on the 
number line, while the second round features the same game 
interactions but involves locating 0.431. Before playing, stu- 
dents did a pretest and answered demographic survey ques- 
tions. After game play, they completed another survey to 
evaluate their experience and did a posttest, followed by a 
delayed posttest one week later. Here we outline the mea- 
sures which are relevant to our analyses. A more detailed 
description of the experimental design can be found in [22]. 


Pretest, Posttest, and Delayed Posttest: Each test 
consisted of 43 items, for a total of 52 points. The items were 
designed to probe for specific decimal misconceptions, and 
involved either the five decimal skills targeted by the game or 
conceptual questions (e.g., “is a longer decimal larger than a 
shorter decimal?”). There are three test versions (A, B and 
C), which are isomorphic to one another and counterbal- 
anced across students (e.g., ABC, ACB, BAC, etc. for pre, 
post, and delayed). Our prior analysis showed no differences 
in difficulty between the three versions [22]. 


Questionnaires: Before game play, students reported their 
age and gender, as well as their ratings to survey items about 
their background information, from 1 (“Strongly Disagree”) 
to 5 (“Strongly Agree”). After playing, students rated their 


enjoyment (also from 1 to 5) via survey questions that ad- 
dress four enjoyment dimensions (Table 1). If a dimension 
comprises several items, we compute the average ratings of 
all items in that dimension to derive its representative rat- 
ing score. According to [16], a measure should have a > .60 
to be considered reliable; therefore, based on Table 1 we re- 
moved the cognitive engagement dimension (with a = .54) 
from further analyses. 


The full log data from the study is archived in the DataShop 
repository [55], in dataset number 3086. We present our 
analysis of this data in the following section. 


4. RESULTS 


4.1 Investigating in-game learning 

In our prior work on Knowledge Component (KC) model- 
ing in Decimal Point, based on data from a separate study, 
we used the correctness of the student’s first attempt in an- 
swering each mini-game problem to update their mastery 
of the KC covered by that mini-game. With this mapping 
from in-game action to KC, we found that students’ learn- 
ing can be better captured by a KC model based on skill 
types (e.g., Addition, Bucket) than on decimal misconcep- 
tions (e.g., Segz, Negz) [40]. Therefore, in this work we used 
the five skill types as our KCs, and tracked students’ learn- 
ing progress of these skills by Bayesian Knowledge Tracing 
(BKT) [60]. The BKT parameters were set as p(Lo) = 0.4, 
p(L) = 0.05, p(S) = p(G) = 0.299 [3], and the mastery 
threshold is 0.9. 


First, we looked at how well students mastered each of the 
five skills in the game. Comparing the students’ final mas- 
tery probabilities in each skill and our mastery threshold, 
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we observed that: there were 4 students who did not master 
any skill, 20 students who mastered one skill, 33 students 
who mastered two skills, 42 students who mastered three 
skills, 34 students who mastered four skills, and 26 students 
who mastered all five skills. Next, we counted how many op- 
portunities each student who mastered a skill took to reach 
mastery in that skill. An opportunity is defined as one com- 
plete decimal exercise; each mini-game round consists of one 
opportunity, except for those in Sequence, which contain 
three opportunities (i.e., students have to fill in three deci- 
mal sequences per round). The distributions of opportunity 
count until mastery are plotted in Figure 2, which shows 
that Number Line and Sorting took the longest to master, 
at around 5 opportunities on average. For Number Line, one 
student even needed 26 opportunities to reach mastery. 


Opportunity count 


Ad 


Addition (60) 


<b 


Bucket (91) Number Line (98) Sequence (131) — Sorting (98) 
Skill 


Figure 2: Opportunity counts until mastery for each 
skill. The number next to each skill indicates the 
count of students who mastered that skill and were 
included in the violin plot. 


Next, we examined how well students regulated their learn- 
ing, i.e., after mastering a skill, did they tend to continue 
practicing the same skill, or switch to a different skill? For 
each student, following [8], once they mastered a skill (> 
90% mastery probability), we considered their subsequent 
opportunities as over-practice. Then, for each student who 
mastered a particular skill, we computed the ratio between 
their over-practice count and total opportunity count in that 
skill. Plotting these ratios for all the mastered students in 
each skill (Figure 3), we observed that between 20-80% of 
a student’s practice opportunities in a skill could be con- 
sidered over-practice, i.e., they took place after the student 
had mastered the skill. 


4.2 Investigating factors related to posttest and 


delayed posttest performance 
Having examined students’ in-game learning, we then looked 
at how it related to test performance after the game. In or- 
der to predict posttest and delayed posttest scores, we col- 
lected features that reflected students’ in-game learning and 
also included demographic measures that account for indi- 
vidual student differences. In total, we considered 19 fea- 
tures: pretest score, decimal efficacy, gender, computer effi- 
cacy, identification agency, intrinsic agency, external agency, 
perseverance, utility, math interest, expectancy, final in- 


game mastery probabilities of the five skills (Addition, Bucket, 


Sequence, Number Line, Sorting), total opportunity count, 
over-practice opportunity count and total incorrect answer 
counts. To identify the most important features, we (1) per- 
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Figure 3: Over-practice ratio in each skilll. The 
number next to each skill indicates the count of stu- 
dents who mastered that skill and were included in 
the violin plot. 


formed feature selection with linear regression, and (2) ran 
another linear regression model with the selected features on 
the full dataset to inspect the coefficient and significance of 
each feature. In step (1), we use the mlxtend library [45] to 
run a forward feature selection procedure that returns the 
feature subset with the best cross-validated performance, 
measured in terms of mean squared error (MSE). 


In predicting posttest scores, our feature selection identi- 
fied three features: Bucket mastery, Sorting mastery and 
pretest score. A linear regression model with these three fea- 
tures, when trained and evaluated on the entire dataset, had 
an MSE of 26.167 and an adjusted R? of .735. Based on the 
regression table, the coefficient and significance of each fea- 
ture was as follow: pretest score with 6 = 0.734, p < .001, 
Bucket mastery with 8 = 6.833, p < .001, Sorting mastery 
with 6 = 5.100, p = .001. In other words, pretest scores, 
Bucket mastery and Sorting mastery each had a positive 
and significant association with posttest scores. 


The delayed posttest model incorporated two additional fea- 
tures — Number Line mastery and gender — and yielded an 
MSE of 24.218, as well as an adjusted R? of .747. Based on 
the regression table, the coefficient and significance of each 
feature was as follows: pretest score with @ = 0.730, p < 
.001, Bucket mastery with G6 = 4.276, p = .018, Sort- 
ing mastery with p = 4.270, p = .003, Number Line with 
B = 3.099, p = .029, and gender with 6 = 1.426, p = .074. 
In other words, the three skill mastery values — Bucket, 
Sorting, Number Line — as well as pretest score each had 
a positive and significant association with delayed posttest 
score, while gender (male = 0, female = 1) had a positive 
and marginally significant association 


4.3 Investigating factors related to enjoyment 
For each enjoyment dimension measured in post-intervention 
surveys (achievement emotion, game engagement, affective 
engagement - see Table 1), we computed the per-student 
average Likert scores to the statements in that dimension. 
Then, we performed the same feature selection procedure as 
in 4.2 and reported our results in Table 2. 


We observed that the adjusted R? values of the game en- 
gagement and affective engagement models were much lower 
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Table 2: Results of feature selection for predicting game enjoyment. The Overall performance row indicates 
the selected model’s scores when trained and evaluated on the entire dataset. 


Achievement Emotion 


Game Engagement 


Affective Engagement 


Selected features | computer _ efficacy, identification 
agency, intrinsic agency, math inter- 


est, pretest score, total opportunity 


math interest, com- | decimal efficacy, gender, intrinsic 
puter efficacy, gender 


agency, Sorting mastery, Bucket 
mastery, total incorrect attempt 


count count, identification agency 
Overall MSE = 0.520 MSE = 0.602 MSE = 0.660 
performance Adjusted R? = 0.386 Adjusted R? = 0.225 Adjusted R? = 0.218 


than those of the test score models. Even when trained 
and evaluated on the entire dataset, Linear Regression could 
only explain about 20% of the variance in game engagement 
and affective engagement. On the other hand, the achieve- 
ment emotion model did have reasonable performance (ad- 
justed R? = .386), so we focused on analyzing the fea- 
tures in this model. The linear regression table showed 
the coefficient and significance of each feature as follows: 
computer efficacy with @ = 0.047, p = .063, identification 
agency with 8 = 0.099, p = .024, intrinsic agency with 
6 =0.116, p = .002, math interest with 6 = 0.114, p = .001, 
pretest score with 6 = —0.017, p = .011, opportunity count 
with 6 = 0.009, p = .033. In other words, computer efficacy 
had a positive and marginally significant association, while 
pretest score had a negative and significant association; the 
remaining features (identification agency, intrinsic agency, 
math interest and opportunity count) each had a positive 
and significant association. 


5. DISCUSSION 


5.1 Investigating in-game learning 

Based on the opportunity count until mastery in each skill 
(Figure 2), we identified Sorting and Number Line as the 
most difficult skills in the game. Our prior learning curve 
analysis [40] on a different Decimal Point study reported a 
consistent finding — that the learning curves of these two 
skills were mostly flat and reflected small learning rates. 
Based on previous research in decimal learning, a plausible 
explanation is that there are several misconceptions which 
can lead to students making a mistake in Sorting or Num- 
ber Line problems, including (1) treating decimals as whole 
numbers, (2) treating decimals as fractions, and (3) ignoring 
the zero in the tenths place [46]. Furthermore, even when 
students recognize their misconception, they may shift to 
a different misconception instead of arriving at the correct 
understanding [56]. This phenomenon likely also occurred 
in Decimal Point, as the game provides corrective feedback 
(whether an answer is right or wrong) but does not empha- 
size the underlying reasoning; consequently, as an example, 
a student realizing it is wrong to assume longer decimals are 
larger may end up concluding that shorter decimals must be 
larger, thereby adopting a new misconception. This high- 
lights the need for more refined tracing of the student’s 
dynamic learning states in a digital learning environment. 
While the standard KC modeling technique can track when 
students make an intended mistake (e.g., longer decimals 
are larger), it does not investigate their specific input to 
see whether a new misconception (e.g., shorter decimals are 
larger) has emerged. To address this issue, future itera- 


tions of the game should provide more instructional support 
that can react to various misconceptions from students, for 
example via explanatory feedback [19] or predefined error 
messages for different types of error [36]. 


Once students have mastered a skill, however, our analy- 
sis showed that over-practice was very common, i.e., stu- 
dents kept playing more mini-games in the mastered skill. 
At the same time, there were only 26 out of 159 students 
who mastered all five skills, suggesting that the majority 
of students still had room for improvement in the unmas- 
tered skills but chose not to practice them. One possible 
reason is that the game environment did not explicitly indi- 
cate when the student has reached mastery or force them to 
switch to practicing a different skill. Consequently, young 
students, who were likely to be weak at self-regulated learn- 
ing [37,53], simply played the mini-games that they thought 
were engaging, which in this case involved the skills they had 
already mastered. A prior study by [29] similarly found that, 
in a game about locating fractions on number line, students 
were more engaged when the game was easier, contradicting 
game design theories that optimal engagement would occur 
at moderate difficulty level. 


5.2 Investigating factors related to posttest and 


delayed posttest performance 

We saw that our linear regression models were able to pre- 
dict posttest and delayed posttest performance well, cap- 
turing about 75% of the variance in test scores with only 
3-5 features. The three features present in both models are 
pretest score, Sorting mastery and Bucket mastery. The 
inclusion of pretest score is not surprising, as it is consistent 
with the standard practice of controlling for prior knowledge 
when analyzing posttest score [58]. On the other hand, both 
Sorting mastery and Bucket mastery suggest that the abil- 
ity to compare decimal numbers plays a large role in test per- 
formance. This is likely due to the game and test materials 
focusing on the four most common decimal misconceptions 
(Megz, Segz, Pegz, Negz), three of which are related to dec- 
imal comparison [25]. Based on the distribution of practice 
opportunities until mastery, however, students took much 
more attempts to master Sorting problems than Bucket 
problems, which may explain why they did not achieve high 
scores on the posttest and delayed posttest, averaging at 
only around 30 out of 52 points [22]. Therefore, improv- 
ing students’ performance on Sorting problems, potentially 
by incorporating hints and error messages as we previously 
discussed, is crucial in future studies of the game. 
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At the same time, we saw that Number Line mastery had a 
significant positive association with delayed posttest score, 
but was not selected in the posttest model. An interpreta- 
tion of this result is that Number Line tasks, which we identi- 
fied as among the most difficult in the game, could be at a de- 
sirable difficulty level, which can promote deeper and longer- 
lasting learning than the more straightforward tasks [61]. 
For instance, a prior study on comparing erroneous exam- 
ples and problem-solving decimal tasks found that erroneous 
examples, which are more aligned with the desirable diffi- 
culty, led to significantly higher delayed posttest scores but 
similar posttest scores [34]. In our case, we also saw that 
Number Line is an important feature for predicting delayed 
posttest but not for predicting posttest performance. 


Similar to Number Line mastery, gender (male = 0, female 
= 1) was not a feature in the posttest model, but had a posi- 
tive association with delayed posttest scores. In other words, 
with other factors being equal, females could achieve higher 
delayed posttest scores than males. While this association is 
only marginally significant (p = .074), similar findings about 
females’ tendency to outperform males in retention and de- 
layed posttest have been reported in previous mathematics 
intervention studies [1,20]. Using the same dataset as in 
this work, [22] also found that females demonstrated signif- 
icantly higher pre-post and pre-delayed learning gains than 
males, with a larger effect size in pre-delayed learning gains. 
Therefore, an important next step is to conduct future stud- 
ies of Decimal Point on a larger sample size to draw more 
conclusive findings about whether the game promotes more 
retention in females and what could lead to this effect. 


5.3 Investigating factors related to enjoyment 
Our enjoyment prediction models did not perform as well 
as the learning models and could explain only about 20% of 
the variance in game engagement and affective engagement. 
These poor model fits likely result from the lack of appropri- 
ate features in our data. To track student engagement, pre- 
vious work has emphasized the use of fine-grained measures 
such as time spent on decision making [47], social engage- 
ment profile [49] and interaction traces [6]; in contrast, our 
feature set consists mainly of quantitative scores (e.g., Likert 
responses) and aggregate data (e.g., error count). Related to 
this direction, a previous study of Decimal Point by [57] has 
clustered students based on their mini-game selection orders 
and found that the cluster which demonstrated more agency 
reported higher enjoyment. Adopting their method of en- 
coding students’ mini-game sequences is a good first step in 
building more fine-grained features for our prediction tasks. 
On the other hand, the lack of association between our in- 
game learning measures (e.g., skill mastery, over-practice op- 
portunity count, error count) and game engagement or affec- 
tive engagement implies that students’ game performance, 
whether good or bad, were unlikely to yield any negative 
emotion such as confusion or frustration. This is a positive 
outcome, indicating that our game environment does not 
impose any performance pressure on students — one of the 
primary principles of learning games [15]. 


At the same time, we did find that a linear regression model 
was able to predict achievement emotion reasonably well 
from student’s identification agency, intrinsic agency, math 
interest, computer efficacy, pretest score and opportunity 


count. Identification and intrinsic agency indicate that, with 
all other factors being equal, the more students identified 
their learning as coming from intrinsic motivation (rather 
than external pressures), the more achievement they felt 
after learning. Math interest and computer efficacy sug- 
gest that students’ acquaintance with the learning domain 
or medium could also be positively associated with achieve- 
ment emotion [26]. On the other hand, pretest score had 
a negative association, likely because students with lower 
prior knowledge were able to learn more from the game 
and therefore felt more achievement than those with high 
prior knowledge. Similarly, for opportunity count, a plau- 
sible reason for students choosing to play more mini-game 
rounds is that they felt the mini-games were helpful, which 
contributed to their achievement emotion after game play. 
Overall, the features we identified could serve as a guideline 
for promoting achievement emotion in learning games and 
in more general instructional contexts. 


6. CONCLUSIONS 


From our analyses, we gained several insights into students’ 
learning outcomes and enjoyment in Decimal Point. First, 
we found that Sorting and Number Line are important skills 
for posttest and delayed posttest performance, but students 
required more instructional support to effectively master 
them. Second, very few students mastered all five deci- 
mal skills from the game, while the majority engaged in 
over-practice, likely due to their preference for playing easy 
mini-games, i.e., those they had already mastered. Third, 
expanding on prior findings about gender effect in Decimal 
Point [22,33], we identified a trend of females outperforming 
males in the delayed posttest, which should be investigated 
on a larger sample size. Fourth, we learned that students’ 
achievement emotion can be reasonably captured by their 
level of computer efficacy, learning motivation, prior knowl- 
edge and number of mini-game rounds. All of these insights 
can be derived from log data alone and would serve as useful 
metrics to assist digital learning game researchers in evalu- 
ating and improving their own games. For Decimal Point, 
in particular, an important next step is to perform similar 
analyses in other studies of the game to see which of our 
findings can be replicated. Identifying consistent trends in 
student data could allow us to construct a more generalized 
model of students’ game play that combines existing theories 
with novel exploratory analyses [38]. 


In a broader context, we have seen the rapid growth of digi- 
tal learning games in recent years, from being conceived as a 
novel learning platform [15,21] to having their effectiveness 
validated by rigorous studies [10]. The game Decimal Point, 
in particular, has been shown to significantly improve stu- 
dents’ learning through several research works [18,22,35,39]. 
When viewing from a learning analytics perspective, how- 
ever, one could identify room for improvement that would 
otherwise not be reflected in pretest and posttest scores 
alone. For instance, a game may not adequately support 
all of its learning objectives, or students may engage in non- 
optimal learning behavior due to a lack of self-regulation. 
At the heart of these issues is the question of how digital 
learning games can optimize student learning while retain- 
ing its core value as a playful environment, where players 
are free to exercise their agency. Addressing this question is 
an important step for future works in the field. 
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